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FOREWORD 


Opinions,  interpretations,  conclusions  and  recommendations  are 
those  of  the  author  and  are  not  necessarily  endorsed  by  the  U.S. 
Army. 

_  Where  copyrighted  material  is  quoted,  permission  has  been 

obtained  to  use  such  material. 

_  Where  material  from  documents  designated  for  limited 

distribution  is  quoted,  permission  has  been  obtained  to  use  the 
material . 

M.  Citations  of  commercial  organizations  and  trade  names  in 
this  report  do  not  constitute  an  official  Department  of  Army 
endorsement  or  approval  of  the  products  or  services  of  these 
organizations . 

_  In  conducting  research  using  animals,  the  investigator (s) 

adhered  to  the  "Guide  for  the  Care  and  Use  of  Laboratory 
Animals,"  prepared  by  the  Committee  on  Care  and  Use  of  Laboratory 
Animals  of  the  Institute  of  Laboratory  Resources,  National 
Research  Council  (NIH  Publication  No.  86-23,  Revised  1985) . 

_  For  the  protection  of  human  subjects,  the  investigator (s) 

adhered  to  policies  of  applicable  Federal  Law  45  CFR  46. 

_  In  conducting  research  utilizing  recombinant  DNA  technology, 

the  investigator (s)  adhered  to  current  guidelines  promulgated  by 
the  National  Institutes  of  Health. 

_  In  the  conduct  of  research  utilizing  recombinant  DNA,  the 

investigator (s)  adhered  to  the  NIH  Guidelines  for  Research 
Involving  Recombinant  DNA  Molecules. 

_  In  the  conduct  of  research  involving  hazardous  organisms, 

the  investigator (s)  adhered  to  the  CDC-NIH  Guide  for  Biosafety  in 
Microbiological  and  Biomedical  Laboratories. 


Abstract 


This  proposal  addresses  the  development  of  a  softcopy  display  for  digital  mammography 
that  seeks  to  couple  optimally  the  visual  system  to  the  displayed  image  without  excessive 
human-machine  interaction.  It  also  breaks  away  from  two  traditional  approaches  to 
workstation  display  design,  namely  trying  to  make  the  softcopy  image  look  like  hardcopy 
and  using  observer  determined  window-level  and  rove-zoom  functions. 

The  display  is  calibrated  by  using  both  the  perceptually  standard  curve  for  the  monitor 
and  an  observer  derived  display  based  upon  the  just  noticeable  difference  in  contrast  over 
the  intensity  range  of  the  monitor.  The  image  is  equalized  to  fit  the  prescribed  intensity 
range.  There  is  also  a  built  in  scanning  procedure  derived  from  eye-position  observation 
on  skilled  mammographers. 

The  calibration  procedures  are  being  tested  and  the  scanning  procedure  is  being  developed. 
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5.  Introduction 


5.1 .  Purpose  of  the  Project 

The  long  term  goal  of  this  research  es  to  improve  the  detection  of  masses  on  digital 
mammograms  by  using  a  softcopy  display  that  will  optimize  the  coupling  of  the  human 
visual  system  to  the  display  without  excessive  human-machine  interaction.  This  coupling 
is  called  a  “perceptually  tempered”  softcopy  display. 

5.2.  Scope  of  this  Project 

This  problem  will  be  approached  by  three  related  aims. 

Aim  1:  Develop  a  method  for  simulating  masses  of  realistic  appearance  on  real 
mammographic  parenchymal  backgrounds  that  can  be  used  as  a  test  set  for  the  subsequent 
development  of  display  methods. 

Aim  2:  Develop  a  method  for  equalizing  perceived  contrast  on  a  mammogram  that  is 
related  to  the  contrast  sensitivity  of  the  reader. 

Aim  3:  Develop  a  computer  controlled  roving  window  based  upon  eye  position 
distribution  patterns  over  mammograms. 

5.3.  Background 

For  the  foreseeable  future,  mammograms  will  be  the  primary  screening  tools  for 
breast  cancer  and  they  will  be  read  by  human  beings.  Whenever  images  are  read  by  hirnian 
beings  there  is  an  associated  error  rate.  Although  the  true  diagnostic  accuracy  for  screening 
examinations  is  unknown,  false  negatives  (misses)  have  been  estimated  to  range  between 
1 1  and  25  %  and  false  positives  have  been  estimated  to  range  between  10  and  35  % . 
These  figures  indicate  that  there  is  room  for  improvement  in  the  readers. 

Digital  detectors  will  be  a  reality  in  the  near  future  making  softcopy  readings 
feasible.  Softcopy  has  many  potential  advantages  but  at  present  is  less  effective  than 
hardcopy.  Simulating  film  with  a  softcopy  display  is  inadequate  because  as  compared 
with  film  currently  available  monitors  do  not  have  sufficient  intensity  range ,  do  not  have 
sufficient  resolution  and  are  too  noisy.  Although  bright  (1000  cd/m^),  high  resolution 
(4000  X  4000  pixel)  monitors  are  feasible,  they  are  too  expensive  for  widespread  use.  The 
window-level  and  zoom-rove  functions  that  are  commonly  used  with  cross-section  CT 
and  MRI  are  not  sufficient  for  mammograms.  Given  the  present  state  of  the  art,  softcopy 
displays  cannot  be  made  that  simulate  film.  Therefore,  it  is  important  to  develop  display 
mode  alternatives  to  the  film-on-lightbox  design  or  cross-section  imaging  design  that  can 
be  used  with  moderate  brightness  (300  cd/m^  ),  moderate  resolution  (2000  x  2000  pixel) 
monitors. 
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Johnson  et  al.  [8.1]  and  Blume  et  al.  [8.2]have  suggested  that  video  monitors 
should  be  “perceptually  standardized”  so  that  equal  changes  in  the  pixel  gray  scale  value 
produce  equal  changes  in  the  just  noticeable  difference  (JND)  of  luminance  in  the  image. 
Although  perceptual  standardization  is  a  worthwhile  way  to  make  monitors  adhere  to  the 
same  standard  input-output  transfer  characteristic,  it  does  not  adjust  either  for  local 
contrast  variations  in  the  image  or  for  the  level  of  light  adaptation  which  is  partially 
determined  by  the  ambient  lighting.  In  order  to  match  the  display  of  a  particular  image  to 
the  visual  system,  the  image  itself  must  be  modified  and  the  display  function  must  be 
adjusted  for  the  adapting  luminance.  In  our  laboratory,  Liu  and  Nodine  [8.3]  using  a  model 
first  proposed  by  Mokrane  [8.4]  have  developed  an  algorithm  that  equalize  perceived 
luminance  over  the  image  assuming  some  starting  level  of  adapting  luminance.  Contrast  is 
modified  in  the  image  on  the  basis  of  the  theoretical  threshold-contrast  curves  of 
Heinemann  [8.5]. 

This  research  extends  the  work  of  Liu  and  Nodine  [8.4]  to  include  calibration  of  the 
individual  observer  and  to  incorporate  perceptually  derived  scanning  strategies  into  the 
search  for  masses  on  mammograms. 

6.  Progress  Report 

6.1 .  Development  of  Simulated  Mass  Lesions  on  Real  Mammographic  Backgrounds 

A  set  of  images  was  selected  fi'om  a  group  of  mammograms  known  to  be  normal 
(stable  for  3  years).  The  images  were  digitized  at  100  microns  resolution  and  square 
sections  about  10x10  cm  in  size  were  removed  fi'om  each  image  and  placed  in  a  file.  A 
computer  program  was  developed  to  simulate  the  appearance  of  the  projection  of  an 
object  onto  the  mammogram  background.  The  fi:ee  parameters  in  the  program  include  the 
size  of  the  ovoid  object  (size  and  shape),  the  linear  x-ray  attenuation  coefficient  of  the 
object  and  the  surround  (contrast)  and  a  geometric  unsharpness  factor  (focal  spot  size  and 
distance).  The  simulator  is  used  by  randomly  selecting  a  section  and  a  point  in  the 
section.  The  section  is  then  displayed  with  the  mass  centered  on  the  selected  point. 

The  simulator  provides  an  inexhaustible  supply  of  very  low  contrast  masses  on 
real  mammogram  backgrounds  for  use  in  testing  the  system. 

6. 2.  Development  of  display  software  and  a  calibration  procedure  to  compensate  for 
several  phenomena  that  may  hinder  the  detection  of  abnormalities  in  mammograms 

6.2.1  Photometric  calibration  of  the  display 

The  input-ou^ut  transfer  characteristic  of  display  monitors  limit  the  number  of 
distinguishable  gray  levels,  even  under  the  most  favorable  of  ambient  room  lighting 
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conditions.  The  assignment  of  available  gray  levels  for  displayed  images  is  optimized  by 
photometric  calibration  followed  by  perceptual  adjustment. 

A  photometer  is  aimed  at  the  center  of  the  monitor  and  the  intensity  of  a  square 
larger  than  the  active  area  of  the  photometer  is  stepwise  increased.  A  plot  of  luminance 
against  the  digital  driving  intensity  is  made.  The  brightness  and  contrast  controls  of  the 
monitor  are  adjusted  to  provide  visualization  of  the  maximum  number  of  gray  levels 
following  the  method  proposed  by  Blume  et  al.  [8.2]. 

6.2.2  Perceptual  calibration  of  the  display 

Psychophysical  measures  on  observers  viewing  subtle  targets  show  that  at  any 
light  adaptation  level  of  the  human  eye,  performance  degrades  when  the  target  surround  is 
sufficiently  different  than  the  light  levels  dictating  the  adaptation  level  of  the  eye.  A 
psychometrically  derived  correction  to  the  display  lookup  tables  is  used  to  boost  the 
displayed  contrast  of  features  that  fall  in  the  image  regions  that  are  photometrically  far 
firom  the  adaptation  level  of  the  observer.  The  calibration  is  done  by  showing  the  observer 
a  set  of  8  small  square  patches  against  a  background  of  gaussian  noise  that  has  about  the 
same  variance  as  a  mammogram.  The  mean  luminance  is  varied  over  8  levels  firom  black  to 
white  and  the  just  noticeable  difference  (JND)  is  recorded.  A  curve  of  JND  as  a  fimction 
of  luminance  is  drawn.  The  curve  is  characterized  by  a  polynomial  which  is  used  to 
modify  the  basic  system  look-up  table  (LUT). 

6.2.3  Image/Display  coupling  optimization 

The  final  step  in  presenting  the  display  is  identifying  the  clinically  significant 
regions  of  the  image  to  maximize  the  number  of  intensities  available  for  target 
discrimination.  These  regions  will  be  determined  from  the  analysis  of  eye  position  data  of 
experts  viewing  mammograms. 

6.3  Develop  a  computer  controlled  roving  window  based  upon  eye  position  distribution 
patterns  over  mammograms. 

The  distribution  of  eye  fixations  over  images  will  be  recorded  in  a  number  of 
expert  mammographers.  The  summed  distributions  will  be  correlated  with  image  features 
such  as  average  intensity  (density) ,  dominant  edges  and  curved  edges.  This  information 
will  be  used  to  develop  search  strategies  for  new  images.  The  strategies  will  be 
implemented  by  a  roving  window  on  the  display. 
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6.4  Results 


6.4.1  Lesion  simulator 

The  lesion  simulator  has  been  developed  and  successfully  used  in  preliminary  tests  of 
visual  contrast  sensitivity. 

6.4.2  Photometric  calibration 

The  photometric  calibration  is  shown  in  Figure  6.4.1 

Figure  6.4.1  Display  Function  Resulting  from  Photometric  Calibration 


0  50  100  150  200  250 

Driving  Intensity 

6.4.3  Perceptual  calibration  under  ambient  lighting  conditions. 

Using  the  photometrically  calibrated  display ,  circular  1  cm.  targets  with  a  range  of  contrasts 
are  displayed  in  known  locations  to  the  observer,  who  chooses  the  targets  that  are  just 
visible.  The  adaptive  level  of  the  observers  eyes  depends  on  both  the  existing  room  lighting 
and  the  overall  brightness  of  the  display.  Ambient  light  affects  contrast  sensitivity  by 
influencing  the  light  adaptation  level  of  the  eye,  by  scattering  off  the  display  surface  and  by 
scattering  widiin  the  eye.  This  reduces  the  perceived  contrast  of  image  features.  The  resulting 
database  determines  the  minimum  contrast  (pixel  driving  level)  required  to  detect  the  target  at 
each  of  eight  background  intensities,  (see  Figure  6.4.2a)  These  data  are  fit  by  a  second  degree 
polynomial  equation,  which  becomes  a  basis  function  for  the  minimal  detectable  contrast  ( 
MDC  )  calibration.  The  relationship  is  modeled  by  a  parabolic  function.  This  function  is 
normalized  to  provide  sufficient  correction  to  render  test  pattern  targets  approximately 
equally  detectable  at  a  given  (pixel  driving  level)  contrast.  The  curve  is  integrated  to  derive  the 
MDC  corrected  LUT  shown  in  Figure  6.4.2b. 
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Figure  6.4.2  a)  On  the  left  are  the  calibration  data  points  and  the  curve  fitting  parabola. 

b)  On  the  right  is  the  look  up  table  (LUT)  that  results  by  integrating  the 
best  fit  parabola. 


MDC  Corrected  LUT 


0  51  102  153  20+  255  0  102+  20+5  3071  +095 

Background  PFxel  Intensity  input  Intensity 

7.  Conclusions 


The  CRT  can  be  simply  calibrated  to  correct  for  ambient  illumination.  Although 
the  calibration  works  with  test  patterns  it  still  has  to  be  tested  using  real  mammograms. 
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Optimizing  the  visualization  of  projection  images  using  a 
photometric,  psychometric,  and  image  based  calibration  procedure. 

Lawrence  C.  Toto,  Harold  L.  Kundel 

Pendergrass  Laboratory* ,  University  of  Pennsylvania,  Philadelphia,  PA  19104 


ABSTRACT 

We  propose  a  method  for  improving  the  visualization  of  potential  target  regions  in  radiographic  projection 
images.  This  is  accomplished  by  modifying  the  intensity  lookup  tables  (LUT)  to  compensate  for,  1)  cathode 
ray  tube  (CRT)  dynamic  range  limitations  and,  2)  for  contrast  sensitivity  characteristics  of  the  human  eye.  We 
match  the  resulting  LUT  to  selected  regions  of  the  image  to  optimize  the  use  of  the  available  intensity  range. 

Keywords:  contrast  sensitivity,  psychometric  calibration,  LUT  optimization 


1.  INTRODUCTION 

Display  technologies  have  evolved  over  recent  years  to  yield  improved  resolution  and  dynamic  display 
intensity  range,  but  they  still  fall  short  of  achieving  the  resolution  and  dynamic  range  visible  on  standard  x~ray 
film.  CRT  technologies  are  producing  displays  that  can  render  images  with  a  dynamic  range  of  about  2.0-2.5 
orders  of  magnitude  (100: 1  to  300: 1)  whereas,  film  is  capable  of  3.0-3,5  orders  of  magnitude  (1000: 1  to 
3000:1).  To  partially  compensate  for  the  limited  range  of  CRT  displays,  observers  are  required  to  vary  the 
window  width  and  window  level.  Details  that  fall  outside  of  the  windowed  intensities  are  usually  clipped  and 
detailed  features  can  be  lost. 

Subjective  impressions  of  observers  often  favor  high  contrast  representations^  as  the  best  rendering  of  an 
image,  even  though  details  are  lost  due  to  intensity  clipping.  These  images  may  appear  more  aesthetically 
pleasing,  but  may  contain  less  useful  information. 

Without  knowledge  of  the  location  of  potential  lesion  sites,  the  window  width  and  level  that  will  render  a 
subtle  lesion  visible  might  only  be  found  by  laborious  trial  and  error.  Moving  the  window  level  and  width  to 
visualize  different  regions  of  an  image  is  a  common  cure  for  dynamic  range  limitations  of  CRT  displays. 

It  has  been  shown  that  the  contrast  sensitivity  of  the  human  eye  is  related  to  the  adaption  level  of  the 
observer  and  the  focal  luminance  surrounding  the  point  of  gaze^.  Therefore,  it  may  be  advantageous  to  initially 
display  an  image  with  an  intensity  lookup  table  (LUT)  that  adjusts  the  image  appearance  to  compensate  for  the 
factors  limiting  the  contrast  sensitivity  of  the  human  observer^ In  the  contrast  sensitivity  curves  shown  in 
figure  1,  the  dashed  curves  illustrate  the  decrease  in  sensitivity  of  the  observer  to  low  contrast  targets  as  the  local 
luminance  deviates  from  the  observer’s  light  adaption  level.  Observers  demonstrate  maximum  sensitivity  to 
low  contrast  targets  when  the  local  luminance  is  close  to  their  adaptation  level. 

Using  this  knowledge,  we  create  a  LUT  that  compensates  for  decreased  contrast  sensitivity  by  selectively 
increasing  the  contrast  at  the  low  and  high  luminance  levels  in  the  image.  The  resulting  nonlinear  correction  to 
the  LUT  results  in  what  we  refer  to  as  a  “tempered”  display. 


Further  author  information  - 
H.L.  Kundel:  Email:  kundel  @oasis,rad. upenn.edu 
L,C.  Toto:  Email:  toto@fovea.pndr,upenn.edu 
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Figure  1. 


2.  METHODS 

The  following  methods  were  developed  on  a  Sun  SparclO  workstation(Sun  Microsystems  Computer 
Corporation,  Mountain  View,  CA).  All  software  was  written  in  Interactive  Data  Language  (Research  Systems, 
Inc.,  Boulder,  CO),  a  high  level  graphics  language  with  the  capability  of  being  run  on  a  variety  of  computer 
platforms  and  operating  systems.  Though  the  display  buffer  is  capable  of  displaying  only  8-bit  image  data,  all 
processing  of  image  data  and  LUTs  occurred  with  12-bits  of  resolution  (4096  gray  levels).  Only  after  all 
mathematical  operations  are  performed  are  the  final  8-bit  images  produced.  This  was  necessary  to  retain  image 
pixel  intensity  resolution  throughout  the  image  manipulation  process. 

A  -  Photometric  Calibration 

So  that  the  display  characteristics  were  standardized  from  one  session  to  the  next,  we  photometrically 
calibrated  the  CRT  using  a  Tektronix  J17  Photometer  with  a  J18(B  luminance  probe  (Tektronix  Inc.,  Beaverton, 
OR).  This  is  a  semi-automated  procedure  that  is  software  controlled.  The  photometer  probe  was  attached  to  the 
face  of  the  display  with  a  suction  cup.  A  central  square  region  on  the  display  measuring  10  centimeters  on  a 
side  was  stepped  through  17  intensity  levels  that  ranged  from  black  to  white.  These  data  were  used  to  generate  a 
polynomial  equation  that  equated  the  driving  level  with  the  photometric  output  of  the  CRT  (figure  2).  The 
photometric  calibration  process  quantified  the  monitor  characteristics.  This  facilitated  setting  the  monitor  to  a 
similar  state  in  subsequent  sessions. 
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Figure  2. 


B  -  Psychometrics 

The  calibration  process  is  simple  and  can  be  accomplished  quickly.  We  calibrated  the  observer’s  contrast 
sensitivity  using  a  “minimum  detectable  contrast”  (MDC)  calibration  procedure  over  a  wide  range  of  available 
intensities.  Circular  1  cm.  targets  with  a  range  of  contrasts  are  displayed  in  known  locations  to  the  observer, 
who  chooses  the  targets  that  are  just  visible  (figure  3A).  The  adaptive  level  of  the  observers  eyes  depends  on 
both  the  existing  room  lighting  and  the  overall  brightness  of  the  display.  Ambient  light  affects  contrast 
sensitivity  in  two  ways.  Firstly,  by  influencing  the  light  adaption  level  of  the  eye,  and  secondly,  by  scattering 
off  the  display  surface  and  reducing  the  apparent  contrast  of  image  features. 

The  resulting  database  relates  the  minimum  contrast  (pixel  driving  level)  required  to  detect  the  target  at  each 
of  eight  background  intensities.  These  data  are  fit  by  a  second  degree  polynomial  equation,  which  becomes  a 
basis  function  for  the  MDC  calibration.  For  our  test  monitor,  the  relationship  is  modeled  well  by  a  parabolic 
function.  This  function  is  normalized  to  provide  sufficient  correction  to  render  test  pattern  targets 
approximately  equally  detectable  at  a  given  (pixel  driving  level)  contrast.  We  integrate  this  curve  to  derive  the 
h&C  corrected  LUT  (figure  4). 

These  psychophysical  measures  show  that  at  any  light  adaptation  level  of  the  human  eye,  performance 
degrades  when  the  target  surround  is  sufficiently  different  from  the  light  adaptation  level  of  the  eye.  The 
psychophysically  derived  correction  was  applied  to  the  display  lookup  tables  to  increase  the  displayed  contrast  of 
features  that  fell  in  the  image  regions  that  are  photometrically  far  from  the  adaptation  level  of  the  observer 
(figure  3B).  Since  there  is  a  limited  dynamic  luminance  range  in  the  CRT  display,  the  gain  in  contrast  in  one 
segment  of  the  luminance  range  will  be  borrowed  from  another  segment.  Thus,  the  resulting  tempered  display 
constitutes  a  compromise  in  the  assignment  of  luminance  levels  to  image  pixel  values.  Although  the  contrast 
is  enhanced  in  the  darker  and  lighter  regions  of  the  image,  contrast  is  reduced  in  those  intensity  range(s)  where 
the  observer  proves  most  sensitive. 


A  -  Target  Test  Pattern 


B  -  Target  Test  Pattern  after  application  of  the  MDC 
Corrected  LUT 
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MDC  Corrected  LUT 
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Figure  4. 


C  -  MATCHING  LUT  TO  IMAGE 

The  resulting  MDC  corrected  LUT  is  matched  to  selected  regions  of  the  image  to  maximize  the  number  of 
intensities  available  for  target  discrimination.  The  regions  are  selected  by  the  user  who  uses  the  cursor  to  define 
a  rectangular  region  of  any  size  containing  the  anatomical  tissue  of  interest.  Extremely  underexposed  (behind 
lead  labels)  and  overexposed  (no  anatomical  tissue)  regions  of  the  image  should  not  be  included  in  the 
optimization  segment.  We  hope  to  include  automated  segmentation  in  the  future. 


A  -  Original  Breast  Image 


C  -  MDC  Corrected  LUT  applied 
to  Selected  Segment 


B  -  Original  Breast  Image  with 
Selected  Segment  for  Optimizing 
Grayscale 

Figure  5. 


In  this  initial  evaluation  phase  of  the  project,  the  software  has  been  written  to  allow  the  observer  to  apply 
the  MDC  corrected  LUT,  return  to  the  linear  function  with  normal  window  level  and  width,  or  utilize  specidly 
designed  nonlinear  LUTs  that  are  based  on  adjustable  general  purpose  gaussian  or  parabolic  basis  functions. 


3.  CONCLUSIONS 

The  minimum  detectable  contrast  of  observers  varies  with  light  adaptation  levels  and  with  display 
charateristics.  We  have  created  an  intensity  adjusted  image  display  using  psychometrically  derived  intensity 
lookup  tables  to  increase  overall  target  visibility  in  test  patterns.  The  MDC  calibration  process  is  quick  and 
simple  to  perform.  Subjective  evaluations  confirm  that  following  this  display  optimizing  results  in  improved 
visualization  of  targets  in  the  test  pattern  images.  We  propose  this  method  for  improving  the  visualization  of 
potential  target  regions  in  radiographic  projection  images.  Observer  performance  studies  will  test  this 
hypothesis. 
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